Field of the Invention
The present invention relates to a technique to correct a disparity map and a depth map.
Description of the Related Art
As the method of correcting a loss of data, a non-uniform error, a pixel value with a low reliability, etc., in a depth map that represents a distance (depth) to a subject by a pixel value, there is filter processing using a smoothing filter. However, the smoothing filter is a filter that reduces the difference in pixel value between a pixel of interest and a peripheral pixel. Because of this, there is such a problem that the boundary portion (edge) of a subject is blurred due to the filter processing. Consequently, as the method of smoothing an image without losing information on an edge, an edge-saving type smoothing filter has been proposed. U.S. Pat. No. 8,253,733 has disclosed the Cross (or Joint) Bilateral Filter (CBF) as an example of the edge-saving type smoothing filter. The Cross Bilateral Filter is a filter that determines a weight to be used for filter processing for each pixel based on the similarity of the luminance value in each pixel of a reference image by preparing an image with a higher image quality (reference image) corresponding to the same scene of an image to be corrected (input image). The Cross Bilateral Filter determines the weight to be used for filter processing based on the reference image that shows a clearer edge than that of the input image, and therefore, it is possible to obtain an image with a clearer edge than that in the case where the weight is determined by using only the input image. Then, by determining the weight to be used for filter processing for each pixel based on the similarity of the luminance value in each pixel of the reference image, it is possible to reduce contribution of pixels corresponding to a different subject, and therefore, it is possible to obtain an image with a clearer edge.
However, with the technique described in U.S. Pat. No. 8,253,733, there is a case where an area occurs where the boundary portion (edge) between subjects whose luminance values are close to each other is blurred. Further, in the scene where the depth value changes continuously also within one and the same subject, there is a possibility that the accuracy of the depth value is reduced due to smoothing. This problem similarly applies not only to the above-described depth map representing the depth to the subject but also to the disparity map representing the distribution of the disparity amount in the entire image from an arbitrary viewpoint. Consequently, an object of the present invention is to improve disparity estimation accuracy while maintaining the edge of a subject in the filter processing of the disparity map and the depth map.